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We present a stability analysis of the 2D t — t' Hubbard model on a square lattice for various 
values of the next-nearest-neighbor hopping t' and electron concentration. Using the free energy 
expression, derived by means of the flow equations method, we have performed numerical calculation 
for the various representations under the point group C41, in order to determine at which temperature 
symmetry broken phases become more favorable than the symmetric phase. A surprisingly large 
number of phases has been observed. Some of them have an order parameter with many nodes 
in k-space. Commonly discussed types of order found by us are antiferromagnetism, ci^2_y2-wave 
singlet superconductivity, d-wave Pomeranchuk instability and flux phase. A few instabilities newly 
observed are a triplet analog of the flux phase, a particle-hole instability of p-type symmetry in 
the triplet channel which gives rise to a phase of magnetic currents, an s*-magnetic phase, a g- 
wave Pomeranchuk instability and the band splitting phase with p-wave character. Other weaker 
instabilities are found also. A comparison with experiments is made. 

PACS numbers; 71.10.Fd, 71.27.+a, 74.20.-z, 74.20.Rp 



I. INTRODUCTION 

In recent years the two-dimensional (2D) Hubbard model has been used 0,0 as the simplest model which maps the 
electron correlations in the copper-oxide planes of high-temperature superconductors since experimental data suggest 
that superconductivity in cuprates basically originates from the Cu02 layers 0. Although in the high-temperature 
cuprate superconductors electron-electron interactions are strong some important features of these systems (in par- 
ticular, antiferromagnetic and d-wave superconducting instabilities) are captured already by the 2D Hubbard model 
at weak to moderate Coulomb coupling. 

Apart from the antiferromagnetism and c?2.2_j,2-wave superconductivity above mentioned (for review see |2, H and 
references therein), a few other instabilities related to symmetry-broken states [sll^ H. lol IToL ITTl IT^ IT^ Il4j . and 
truncation of the Fermi surface in the 2D t — t' Hubbard model with next-nearest-neighbor hopping t' have 
been reported. One of these phases is the staggered flux phase of singlet type with d-wave character which occurs 
in the particle-hole channel and breaks translational and time-reversal symmetries. The singlet flux phase has been 
discovered in the mean-field treatments of the 2D Hubbard model by Kotliar and independently by Affleck and 
Marston j^. It has been discussed by Nayak 16] and Chakravarty et al. 0| as a c?-density wave state which rnay 
coexist with d-wave superconductivity. Recently, a triplet analog of the staggered flux phase has been observed [lj| 
also in the 2D t — t' Hubbard model in certain regions of the parameters. 

Another instability in the 2D t — t' Hubbard model is a c?-wave Pomeranchuk instability breaking the tetragonal 
symmetry of the Fermi surface, i.e. a spontaneous deformation of the Fermi surface reducing its symmetry to 
orthorhombic. It has been recently observed for small values of t' from renormalization group calculations by Halboth 
and Metzner They argued that the Pomeranchuk instability occurs more easily if the Fermi surface is close to 
the saddle points of the single particle dispersion (Van Hove filling) with a sizable t' (reducing nesting which leads 
to antiferromagnetism). However, within their technique it is difficult to compare the strength of the Fermi surface 
deformation with other instabilities and to conclude which one dominates. The authors of Ref. have investigated 
the interplay of d-density wave and Fermi surface deformation tendencies with those towards d-wave pairing and 
antiferromagnetism by means of a similar temperature-flow renormalization group approach. They have found that 
the d-wave Pomeranchuk instability is never dominant in the 2D t — t' Hubbard model (even under the conditions 
mentioned above). At the same time, the d^^ -y-i-viave Pomeranchuk instability has been observed [T^ to be one of 
the strongest instability in the 2D Hubbard model by means of the flow equations method, but these calculations 
have some limitations at low temperatures (see Section Hi)) . 

On the other hand, VoUhardt et al. |1S] showed that t'-hopping destroys antiferromagnetic nesting instability at 
weak interactions in two and three dimensions, and supports the stabilization of metallic ferromagnetism in infinite 
dimensions away from half-filling. Therefore, one may expect also the stabilization of ferromagnetism by a sizable t' in 
two dimensions. Indeed, in the t — t' Hubbard model on a 2D square lattice at week to moderate Coulomb coupling, 
a projection quantum Monte Carlo calculation with 20 x 20 sites and the T-matrix technique 0, a generalized 
random phase approximation including particle-particle scattering 8] point towards a ferromagnetic ground state 
for large negative values of t' in a density range around the Van Hove filling. Similar tendencies have been found 
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by the authors of Ref. [ICl within the renormahzation group and parquet approaches. Honerkamp and Salmhofer 
recently studied pj| the stabiUty of this ferromagnetic region at finite temperatures by means of the temperature-flow 
renormahzation group technique. They have found that ferromagnetic instability is the leading one at t' < — 0.33t 
and the Van Hove filling with critical temperatures depending on the value of t' . When the electron concentration 
is increased slightly above the Van Hove filling, the ferromagnetic tendencies get cut off at low temperatures and a 
triplet p-wave superconducting phase dominates. However, they did not consider the Pomeranchuk instability and 
other ones apart from antiferromagnetism, d- and p-wave superconductivity and ferromagnetism. 

Other instabilities recently found in the 2D Hubbard model are an "insulating spin liquid state" [l5l | and "band 
splitting" phase |l3j | . Insulating spin liquid phase is the state with truncated Fermi surface and unbroken translational 
symmetry. It is insulating due to the vanishing local charge compressibility, and a spin liquid because of the spin gap 
arising from the pairing correlations. Band splitting phase is a particle-hole instability of singlet type with p-type 
symmetry which has been found close to half-filling. It leads to a splitting into two bands, and may lead to an 
energy gap in the charge excitations spectrum. 

In the present paper, which is the continuation and extended version of our previous work |14| . we investigate 
the possible phases of the 2D t — t' Hubbard model on a square lattice with the help of the free energy expressions 
derived in Ref. p3i | by means of the flow equations method. In addition to the short paper we present a summary 
of the formalism, and give some details of numerical calculations as well as consider the case of arbitrary electron 
concentrations. The phases found by us are discussed in more details. We find all commonly discussed states as well 
as a few new possible phases. We determine in which regions various symmetry broken phases are more favorable 
than the symmetric (i.e. normal) state. This is done for various types of symmetry breaking independently, that is if 
no other symmetry breaking would be present. One phase may suppress another phase. To which extend two order 
parameters can coexist with each other is a question, which has to be investigated in the future. The paper has the 
following structure. In Section^we summarize the formalism, and represent the details of numerical calculations. In 
Section fllll the numerical results of our stability analysis are given, and the possible phases of the model are discussed. 
Section Hvl is devoted to the conclusions. 



II. FORMALISM AND NUMERICS 



We consider the t — t' Hubbard model 

H = ^£k4<xCk<T + J^Y. ^(ki,k2,k'i,k'2)4^^ck'j4^^ck^^, (1) 

where £k is the Bloch electron energy with the momentum k, Ckg.(cko-) is the creation (annihilation) operator for 
the electrons with spin projection a G {'1,1}, and initially V^(ki, k2, k'j^, kj) ~ t^<^ki+k2,k'j+k^ is the local Coulomb 
repulsion of two electrons of opposite spins, N is the number of lattice points, lattice spacing equals unity. 
For a square lattice the single particle dispersion has the form 

£k = — 2t(cos fcj, + cos ky) — At' cos cos ky, (2) 

where t is the hopping integral of electrons between nearest neighbors of the lattice, and t' is the next-nearest-neighbor 
hopping integral. The spectrum (PJ contains Van Hove singularities in the density of states at the energy evh = 4t' 
related to the saddle points of the Fermi surface at k = (0,±7r) and (±7r,0). For t' — and half-filling the Fermi 
surface is nested ek+q = ~£k with Q — (tt, tt), which leads to an antiferromagnetic instability for U > 0. The nesting 
is removed for t' /t ^ 0. 

We are mainly interested in the following channels 

V^B(k,q) =^(k,-k,q,-q), (3) 

l^ff(k,q) = V^(k,q,k,q), (4) 

Fj^(k,q) =y(k,q,q,k), (5) 

l^A(k,q) =F(k,q + Q,q,k-HQ), (6) 

Fc(k,q) = F(k,q + Q,k + Q,q), (7) 

W(k,q) = l/(k,Q-k,q,Q-q). (8) 



Vb describes the coupling of electron-pairs, V// and Vp are interactions showing up in Fermi-liquid theory and 
describe (homogeneous) magnetism and Pomeranchuk-instabilities, channels Va and Vp describe antiferromagnetism 
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TABLE I: The factors ea,eb, and r in Eg. [101 



J Ea Eb r 

B Ek-Eq Eq-ek k + q 

F Ek + Eq Ek + Eq k + q 

H Ek - Eq Ek - Eq k - q 

A Ek + Eq+q £k+Q + Eq k + q + Q 

C Ek — Eq Ek+Q — Eq+q k — q 

Y Ek-Eq-q Eq-£q_k k + q-Q 



and charge density waves with wave-vector Q as well as some other types of ordering. Vy describes electron pairs 
with wave- vector Q. These channels are kept within our approach, but finally the effective interaction Vy is rather 
weak. We apply the flow equation method jiol l20l | to this Hamiltonian, that is we perform a continuous unitary 
transformation as a function of a flow parameter I which brings the Hamiltonian into a mean-field form, 

^ = [^{l),Hil)], rj{l) = [Hil),H'\l)]. (9) 

Tjil) is the generator of the unitary transformation. H^{1) is obtained from the interaction part of the transformed 
Hamiltonian H{1) by T^''(ki, k2, k'j^, kj, ^) = reii(ki, k2, k'^, k2)V'(ki, k2, k'j^, k2, ^), where Teu is the elimination factor 
which indicates how urgently F(ki, k2, k'j^, k2, 1) should be eliminated. We choose rcii(ki, k2, k'j^, kj) = 'Ylai'^^i +''^£2 ~ 
v^, — v^i )^ with = — = I'k+Q- This condition guarantees, that those interactions we wish to keep have rdi = 
and are kept, but the other ones have > and thus will be eliminated. In this way we eliminate the fluctuations 
around the molecular-field type behavior and keep finally an interaction for which the molecular-field treatment is 
exact. 

Keeping these interactions in second order in the Hubbard coupling U by means of the flow equation method po| 
the Hamiltonian is transformed into an effective one of molecular-field type which contains products of the biquadratic 
terms c^c, c^c^, and cc with total momenta and Q, and depends only on two independent momenta. The second order 

effective interactions are V^^^ = Wb, vj,^^ = Wp, V^^ = V^^ + Wh, vf^ = Wa, = + ^c, Vy^ = Wy, 
with 

_ (rip^ - npj(£p^ -I- £p, - [£a + £b]/2) , , . 

where i = B, F, H, A,C,Y, rik is the Fermi distribution function, fip^ = 1 — np^, ip^ = Sp^ for i = F, A, and 
^P2 = '^P2; £pi = ~£pi foi' i = B, H,C,Y respectively 0. The factors ea,eh, and r are hsted in tabled Then, the 
free energy has the form 

kq ^ ^ k kq ^ ^ 

where the first term of the middle part of Eq. pi|l is the energy contribution and the second term is the entropy 
contribution, and 

_ vif3t _ /3tT4,q f . 

with (3 — l/(fcsr), T is the temperature, Vk,q is effective second-order interaction (the factor U'^ /t has been extracted 
from it), vi = ±1 is the sign with which the first order contribution enters into the effective interaction, /k is an 
entropy coefficient, and Ak are the order parameters. For example, (ckcrC-kcr') = {o'y)crcr'A^ + J2ai'^y'^a)a-a-' Al^ , 
where ctq is a Pauli spin matrix (a = x,y,z), and (A^") is the singlet (triplet) amplitude. An expression similar 
to Eq. pi|l is obtained for particle-hole channels with the order parameters v instead of A. In this case, for example, 

we have (cj^^Ck+Qo-') = K^cr,a' + J2a V^{(^a)aa'- 

In total we consider four channels, two particle-hole and two particle-particle ones, since the total momentum can 
be and Q. In all cases we distinguish between singlet and triplet excitations, and in the channels corresponding 
to the total momentum Q we distinguish also between Ak+q = ±Ak and t'k+q — ii'k respectively. Therefore, we 
introduce the following notations: ph denotes particle-hole channel, si [tr) denotes singlet (triplet), q± corresponds 
to Ak+Q = ±Ak and t^k+q — i^^k respectively, otherwise 0. 
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In Eq. and the analogous expressions for particle-hole channels one has 



channel 




'm^ t"7 / Tf 11 
pp i I LI U 




ph si 


+ 1 


ph tr 


-1 


pp si/tr q± 


+ 1 


ph si q± 


+ 1 


ph tr q± 


-1 



The entropy coefficient function is 



V^(k,q) /(/3(£k-/i),/3(A^~£k)) 
2vj,'\k, q) - Vj,'\k,q) /(/3(ek - /i), /3(£k - m)) 

-y^')(k,q) f^p(eu-^^),P{e^~^l)) (13) 

T4')(k,q) /(/3(ek-M),/3(M-ek+Q)) 

2\4') (k, q) - Vi'^ (k, q) /(/3(ek - /^), /3(ek+Q - m)) 

-yi'^(k,q) /(/3(ek-Ai),/3(£k+Q-M))- 



/(x,2/) = ^^(e- + l)(e^ + l). (14) 
e — e'' 



We start from the symmetric state and investigate whether this state is stable against fluctuations of the order 
parameters A and v. As soon as a non-zero A or v yields a lower free energy in comparison with the symmetric state 
with all vanishing A and i/, then the symmetric state is unstable and the system will approach a symmetry broken 
state. This indicates a phase transition. Of course, one phase may suppress another phase. To which extend two 
order parameters can coexist with each other is a question, which has to be investigated. But this problem lies outside 
the scope of the present paper. 

We performed numerical calculation on a square lattice with 24 x 24 points in the Brillouin zone for the various 
representations under the point group C41, = Amm. This group consists of the following symmetry elements: mirror 
reflections with respect to the x- and y-axes and the lines y = ±x, a fourfold rotation about the axis which is 
perpendicular to x-y-plane. The representations of the even-parity states are one-dimensional. We denote them by 
s+ = si, s^ — g — Sxy(x'^-y'2), fi+ = dj.2_y2^ o?_ = dxy Thc odd-parity representation is two-dimensional, here 
simply denoted by p. 

Initially, such numerical calculations have been performed in Refs. 0,1^, but they were sensitive to the lattice 
size at low temperatures, specially for the particle-hole channels. The main problem is that the wave-vector space 
contributing essentially to the order parameter is very small at low temperatures as a result of exponential divergence 
of the entropy coefficients /k. As we consider the stability of the symmetric state with respect to a symmetry broken 
phase (thus we are interested in small fluctuations of free energy around zero) and the energy contribution to 
Eq. Hll|l changes slowly, the entropy contribution has to change slowly also. Rewriting Eq. in terms of new 
renormalized order parameter Ak = v^^k, we conclude that the averaging (-^) has to be performed in k-space 
when we are calculating entropy coefficients. For the value of the inverse entropy coefflcient l//ko we take the average 
value of the coefficients over Na x Na lattice points of a grid with center at the kg-point and the spacing 27r/ (NaVN). 
The number Na of points in momentum space is inversely proportional to temperature and ^/N, all together these 
points cover the Brillouin zone. A similar procedure is applied to the chemical potential calculation. One should 
also perform the averaging in determination of effective interactions, but it would take much more time. We have 
checked the size effects in the framework of such an improved scheme, performing some calculations also for 16 x 16 
and 32 x 32 lattices, and found that the differences are very small and unessential. Although size effects increase at 
low temperatures in the low density region, they do not touch and change the leading instabilities essentially. 

For each channel the eigenvalues of the matrix -|- have to be determined. Whenever the lowest eigenvalue 
A (i.e. the most negative) equals -1, then a critical Tc/t or {U/t)c is reached. For the s+ representation (in case of 
the effective interactions Va, Vc, Vy only for fk-i-q = +i'k, ^k+q = +Ak) one has to find the solution by iterating 
the eigenvalue equation as a function of U/t. In the case of all other representations the A-term does not contribute. 
Therefore then we determine the lowest eigenvalue A of the matrix B, and obtain {U/t)c = l/\/— A. 



III. POSSIBLE PHASES OF THE MODEL 



We start from i' = and half-filling n = 1 (see Fig. . As expected in this case the leading instability is the 
antifcrromagnetic one, it appears in the ph tr channel with q^ symmetry and s+ wave character. As we leave half- 
filling the antiferromagnetism becomes weaker (Fig.EJ and disappears at hole doping (5 = 1 — n ~ 0.08. As soon as 
the system is doped the nesting is reduced, since there are no longer points on the Fermi surface connected by the 
wave-vector Q and we observe a reentrant behavior. The second order contribution to the antifcrromagnetic channel 
suppresses antiferromagnetism (at least in the s-channel). Therefore antiferromagnetism disappears at larger values 
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of U /t. Strong coupling calculations yield a. t — J- model in which the exchange coupling decreases like J = At^ /U 
for large U . Thus, our observation is remarkable, since we work with a weak-coupling calculation, we obtain at 
intermediate couplings the same tendency as it is expected at strong interactions although we do not reproduce the 
Neel temperature behavior ~ /U . We take the decrease of the Neel temperature as an indication that the 
calculation in second order in U yields reasonable results even for intermediate values U ^ At. For stronger couplings 
higher order contributions will become important. However, it is of interest to see which instabilities emerge within 
our approximation for larger values of U /t, since this gives a hint which types of ordering should be investigated 
for stronger couplings. Therefore, we discuss the phase diagram obtained from the second order calculation also for 
larger values U /t. It might be also that higher order contributions neglected by us enhance the tendency towards 
antiferromagnetism, and then it will decay not so rapidly. Nevertheless, we emphasize that already second order 
calculations give the qualitative description of antiferromagnetism at intermediate couplings. In the latter context, 
we make a comparison of our results with experimental data on some high-temperature cuprate superconductors and 
transition metal compounds. As one can see from Fig. ^ antiferromagnetism disappears at the temperature T « O.li. 
Taking the estimation for the hopping integral t m 0.1 0.3 eV, which is the realistic values for transition metal 
and their compounds 22, "23], we obtain the Neel temperatures Tc = Tn ~ 110 350 K. These values agree with 
experimental data on half- filled antiferromagnetic materials p^ . Let us consider also the parent compound La2Cu04 
of high-Tc cuprate systems La2_2;M2:Cu04 {M = Sr, Ba, Ca). It is an antiferromagnetic insulator with the Neel 
temperature Tjv ~ 300 K and half-filled d-band The electronic structure of these systems is considered to be 

described by the three-band Hubbard model (or d — p model) ^3 • Some of the model parameters deduced from 
photoemission and LDA calculations are Udd ~ 6 10 eV, and tpd ~ 1 1.5 eV 0]. The main features of the 
three-band Hubbard model can be mapped onto the single-band Hubbard model, and therefore one may describe 
these compounds by the Hubbard model The values of the parameters U and t of such a single-band Hubbard 
model are smaller in comparison with the corresponding parameters Udd and tpd of the three-band extended Hubbard 
model, the Coulomb repulsion U is specially reduced (221]. Then, taking the values t « 1 eV and U/t ^ 4.7 4.8 (it 
corresponds to Tc/t w 0.03 ~ 0.02) we obtain the theoretical estimation of the Neel temperature for La2Cu04 around 
Tm « 290 K, that is very close to the experimental value. 

The next instability is a Pomeranchuk instability with d^^-y^ -wave symmetry in the singlet channel (namely ph si 0). 
The corresponding eigenvector signals a deformation of the Fermi surface which breaks the point group symmetry 
of the square lattice. At high temperatures the system has a tetragonal structure and an orthorhombic one at low 
temperatures. The dj.2_y2 wave Pomeranchuk instability dominates in the region of temperature T > 0.05i at small 
hole doping {5 < 0.2), and for negative t' > —t/3 the Pomeranchuk instability dominates in all temperature region 
at the electron concentrations around the Van Hove filling (see Figs. 13 EJ- It competes with other instabilities at 
t' < —t/3 and the electron concentrations around the Van Hove filling, and it is not the leading one here (Fig.lSJ. But 
the d-wave Pomeranchuk instability appears again to dominate at large negative t' < —t/S in the region of electron 
density above the Van Hove fiUing and below some doping (for example, see Fig.E)); for the value t' = —5t/12 this value 
of doping equals (5 « 0.2. In agreement with the ideas of Ref. the instability is mainly driven by a strong attractive 
interaction between particles on opposite corners of the Fermi surface near the saddle points and a repulsive interaction 
between particles on neighboring corners. To favor such a behavior we need a sizable t' reducing antiferromagnetic 
correlations. The critical temperature of the d-wave Pomeranchuk instability occurrence decreases with increasing 
hole doping, and at sufficiently large values of doping and away from Van Hove filling this phase requires strong 
couplings. Thus the hole doping suppresses the tendency towards an orthorhombic distortion of the Fermi surface (or 
lattice). The tendency towards a spontaneous deformation of Fermi surface with d-wave character has been observed 
by A. P. Kampf and A. A. Katanin to be one of the leading instabilities in the 2D extended U-V-J Hubbard 
model in the parameter range which is relevant for underdoped cuprate systems. A similar Pomeranchuk instability 
has been found by the authors of Ref. [2^ to be a characteristic of a nematic Fermi fluid [s^ in two dimensions. 

Another possible phase is a particle-hole instability of singlet type with staggered p-wave symmetry (ph si p) . 
The corresponding quasiparticle energy spectrum is 



varies like ixSinkx -\- iy sin ky with roughly constant ix and iy. It yields a splitting into two bands and may lead 
to an energy gap in the charge excitations spectrum. Another mechanism for a charge gap formation has been 
proposed pjlL l32| recently in the 2D Hubbard model with = at weak coupling. The band splitting phase is 
developed in the region of electron concentration around half-filling, and is one of the strongest in that region. This 




(15) 



where 




(16) 



q 
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instability dominates when the electron concentration n > 1.10 and t' (Fig. [7J. The negative next-nearest- 
neighbors hopping favors this phase to be dominant instability at electron doping, but the value of coupling strength 
U/t required for the occurrence of this instability increases with increasing \t'\. The critical temperature of the 
transition to the band splitting phase decreases with increasing electron doping, thus the electron doping suppresses 
the tendency towards this phase. 

The singlet superconducting dx2_y2 instability (pp si d+) coincides with the dx2^y2-wave staggered flux phase 
at half-filling and t' — (the discussion on flux phases appears in the next paragraph). Away from half- filling the 
degeneration disappears, and d-wave superconductivity dominates at low temperatures in certain regions of electron 
concentration around half-filling which depend on the value of t' ^ 0. Even large values of \t'\ do not destroy the 
dominant low-temperature behavior of drp._y2-^QM% superconductivity at hole doping (Fig.QJ- It is not destroyed also 
at sufficiently large hole doping: for small t' > —t/6 even at the values of hole doping 6 ~ 0.3, and for t' < —t/3 at 
S ^ 0.15 the d-wave superconductivity dominates at low temperatures. However, this dominant behavior is destroyed 
around the Van Hove filling where other phases appear to dominate. At small electron doping d-wave superconductivity 
dominates at low temperatures also, but it is not the leading instability at n — 1 > 0.1 and t' ^ (see Fig. Ejl. In 
contrast to the pure 2D Hubbard model with t' — 0, an electron- hole asymmetry of the d-wave superconducting phase 
is manifested in the t — t' Hubbard model. In this model with negative t' the d-wave superconductivity appears to 
dominate within wider region of hple doping in comparison with the electron doping in agreement with the one- loop 
renormalization group analysis 33]. In this connection it should be mentioned that the most examples of discovered 
high-temperature superconducting materials are hole-doped systems, and only a few examples are electron-doped 
ones. 

We observe also particle-hole instabilities with staggered symmetry of d+ wave character in singlet and triplet 
channels {ph si/tr d+). These states are singlet and triplet flux phases which are approximately described by the 
operators structures 

i ^(COS - cos fcy)Ck+Q^Ck<T, (17) 
kcr 

i ^(cOSfca; - C0Sfcj^)(Tcrcr'4_|_Q^Ckcr', (18) 

respectively, where the components of a are 2x2 spin Pauli matrices. For t' = critical temperatures of the transition 
to the singlet and triplet flux phases are degenerate, in this case these phases vanish at hole doping 6 ^ 0.12 in 
agreement with a mean- field solution of the t — J model presented in Ref. For t' ^ they occur more easily 

at the electron concentrations which are slightly above the Van Hove filling, but the singlet flux phase disappears 
for t' < —t/3. For non-zero values of the next-nearest-neighbors hopping the critical transition temperatures of 
these phases are different, and the triplet one is higher. Moreover, the triplet analog of flux phase dominates at low 
temperatures and t' = — 5i/12 when the electron concentration is slightly above the Van Hove filling (see Fig.lS)) in 
contrast to the results of Ref. |llj| which point out the occurrence of triplet superconductivity with p-wave symmetry 
in this region. The triplet flux phase is also one of the leading instabilities for various values of next-nearest-neighbors 
hopping t' and certain region of electron concentrations (for example, see Fig. |31|Slinj. 

The singlet flux state, described by Eq. H17|) . breaks time-reversal, translational, and rotational symmetries. This 
state is a phase of circular charge currents flowing around the plaquettes of a square lattice with alternating directions. 
The authors of Ref. [s^l found that these currents generate a small magnetic field, also forming a staggered pattern, 
and they estimated the field strength just above the plaquette center to be around 10 G. They pointed out that 
muon-spin-rotation experiments may be able to detect this magnetic field. Recently, the idea of circulating orbital 
currents (also called an orbital antiferromagnetisni) has been discussed as promi sing candidates for a description of 
the pseudogap phase of high- Tc cuprates 0, [s^ [s^. There is also a proposal [37| of a circulating current phase, 
which does not break translational symmetry. The possible existence of orbital charge currents has most recently 
received experimental support. Angle-resolved photoemission with circularly polarized light identified intensity dif- 
ferences for left- and right-circularly polarized photons in the psedogap phase of Bi2Sr2CaCu208+5 H^. Also small 
c-axis oriented ordered magnetic moments, and an evidence for static alternating magnetic fields below the psedogap 
temperature in the underdoped YBa2Cu306+2^ systems were observed by neutron scattering ^9'| and muon-spin- 
rotation [40l | measurements respectively. These data can find a natural interpretation in terms of planar circulating 
current phases [Sj . 

The triplet flux phase, described by Eq. (|18|l . breaks translational, rotational and spin-rotational symmetries, and 
does not break time-reversal invariance. This phase is a state in which spin-up and spin-down electron currents circu- 
late around the plaquettes in opposite directions to produce non-zero spin currents. It corresponds to an alternating 
pattern of spin currents. Similar triplet flux phase (so called spin flux phase or spin nematic state) has been discussed 
by Narsesyan and co-workers ^3 by means of a mean-field theory, and more recently in Refs. |2^ by means of 
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renormalization group approaches in the framework of the 2D extended U-V-J Hubbard model. Another triplet flux 
phase, which breaks time-reversal invariance, has been considered by Nayak as a triplet analog of the density 
wave order parameter potentially relevant to the cuprates. To our knowledge a triplet version of the flux phase has 
not yet been observed in numerical solutions of the 2D t — t' Hubbard model. In the case of polarized particle-hole 
pairs (it means that we take Oz instead of a in Eq. (|18|l ) the alternating pattern of spin currents will generate electric 
field, which could be, in principle, measurable in experiments. As triplet flux phase H18|l does not have anomalous 
expectation values for the spin density but, rather, for spin currents, these excitations do not contribute to the spin- 
spin correlation function, therefore spin currents do not couple to photons, neutrons or nuclear spins 16]. However, 
they could be detected with nuclear quadrupole resonance or two-magnon Raman scattering experiments |l6j . 

At t' = — 5i/12 a few other new instabilities appear to compete at the Van Hove filling and low temperatures 
(Fig. O in disagreement with the conclusions of Ref. fTl| on the occurrence of ferromagnetism. The leading one is 
another Pomeranchuk instability in the s+ channel with g+ = gxi^yi_Qx-iyi wave character {ph si s+, 4 node lines 
in k-space). This phase occurs more easily when the electron concentration is close to the Van Hove filling, and it is 
a leading instability at the Van Hove filling and the values of electron concentration which are slightly smaller than 
the Van Hove filling (Figs.EHHI). The g+ Pomeranchuk instability appears at sizable values of next-nearest-neighbors 
hopping t' ^ —t/i (Fig.|3Jl, but it requests sufficiently large absolute values of t' [t' < —t/3) to be one of the dominant 
instabilities. In the (i_ channel an i-wave (6 node lines in k-space) Pomeranchuk instability (jih si d-) appears 
when electron concentration n is smaller than the Van Hove filling at t' = — 5t/12 (Fig.O. It is a leading one at small 
values of the electron concentration fFig. I10|l . and dominates in the temperature region T > 0.15t at t = —t'/i when 
the electron concentration is sufficiently below the Van Hove filling. The increase of electron concentration reduces 
essentially this phase. 

When the electron concentration is decreased below the Van Hove density at t' — ~5t/12, a particle-hole instability 
of p-wave symmetry in triplet channel (ph tr p) dominates (see Fig. I1(J|I . The electron operators structure which 
describes this magnetic phase is roughly 

i (if Sinfc, + sinfc,)a,,,cLck.' - ^ (^f - 4-e.,J + if [4+e„. " 4-e„j) '^..'CH,.' (19) 

kfTfr' Flcrcr' 

with (cy) being the unit vector along the x- (y-) axis, R denotes lattice site. The quantity is given by 
i*^sinfca; +ty^ sinky fa ~'i-/NJ2q^F(^,<l)'^q- One can see from Eq. ((T^ that this instability gives rise to a phase 
of magnetic currents, the magnetization equals zero as a result of p-wave character of the instability. The magnetic 
currents phase dominates also at very low temperatures and electron concentrations which are slightly smaller the 
Van Hove filling for t' = — 5t/12 (Fig. O, and in the temperature region T < 0.15t below the Van Hove density for 
t' = -t/3. 

In the temperature region T > O.lSt a particle-hole instability with s*-wave character (ph tr s+, its order 
parameter changes sign close to the Fermi-edge) in the triplet s+ channel dominates at t' < —5t/12 and the density 
range around the Van Hove filling (see Figs. [S] |S1 EJl. It is likely that the order parameter contributions do not 
compensate exactly, so that a weak ferromagnetism appears. When the electron concentration is increased above the 
Van Hove filling this instability does not become weaker, but the dx^^y^ wave Pomeranchuk and the triplet flux phase 
instabilities are stronger and dominate at low temperatures. Then, this s*-magnetic phase disappears at sufficiently 
large values of electron concentration in comparison with the Van Hove filling, or smaller \t'\. With the decrease of 
electron concentration from the Van Hove filling this instability becomes weaker and disappears to be a leading one 
at small electron density fFig. I10|l . From Figs. |S1 |H1 Elone can see a reentrant behavior of the s*-magnetic phase in 
some region of the values U/t: approaching we get Tj: from low temperatures and T" from high temperatures at 
the same value of coupling U/t (T^ ^ T"). This is a result of different behavior of Tc(U/t) in two regimes. First 
regime occurs in the region where the s*-magnetic instability dominates and the transition from a paramagnetic 
state to the s*-magnetic phase occurs directly without any intermediate phase, it corresponds to the temperatures 
T > O.lSt in Figs. [Sj 121 In this case the critical temperature increases with the increase of correlation strength U/t. 
Therefore electron correlations enhance the tendency towards the s*-magnetic instability. Another regime occurs at 
the temperatures T < Q.15t, where the s*-magnetic phase is not a leading instability and other instabilities dominate. 
In this situation the critical temperature exhibits an anomalous behavior, it decreases with increasing the coupling 
U/t. The s*-magnetic phase is reduced. Since at lower temperatures only a smaller region in k-space around the 
Fermi-edge contributes, the sign-change of the order parameter reduces the effective interaction. As the transition to 
this phase requires some finite value of the effective interaction at low temperatures, the correlation strength has to 
increase rapidly. 

We observe also a few other weaker instabilities. Apart from d.j.2_y2 wave superconductivity mentioned above 
other two superconducting instabilities appear. At t' < —t/3 a d^y wave superconducting phase occurs in the singlet 
channel sufficiently below the Van Hove filling (see, for example. Fig. I10|l . and for larger absolute values t' < ~bt/12 
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we observe a g+ wave superconductivity in the s+ singlet channel below the Van Hove filling (Fig.lSj). All these phases 
require strong couplings. 

The peculiar feature of the superconducting phase in comparison with the other phases observed by us should be 
noted. Away from the Van Hove filling when temperature approaches zero the curves corresponding to superconducting 
phase are flat, whereas the curves corresponding to all other phases observed become steep. Therefore, at very low 
temperatures the transition from a paramagnetic phase to the superconducting one can occur at very small values of 
the corresponding effective interaction in contrast with the transitions to other possible phases which require some 
finite values of the effective interactions. One can see also that the critical temperatures of all phases (excepting 
antiferromagnetism and s*-magnetic phase in certain regions) increase with the increase of correlation strength U /t. 
Thus, electron correlations enhance the tendency towards the transition to the phases observed by us. 

IV. CONCLUSIONS 

We have presented a stability analysis of the 2D t — t' Hubbard model on a square lattice for various values of the 
next-nearest-neighbors hopping t' and electron concentration. On the basis of the free energy expression, derived by 
means of the flow equations method, we have performed numerical calculation for the various representations under 
the point group C/^^, in order to determine the phase diagram. A surprising large number of phases has been observed. 
Some of them have an order parameter with many nodes in k-space. 

For small values of t' and doping the leading instability is the antiferromagnetic one. At t' ~ Q antiferromagnetism 
disappears at hole doping 5 ~ 0.08 and temperatures T ~ O.li. This result reproduces the Neel temperatures in 
some antiferromagnetic materials. The (1^.2 _y2 wave Pomeranchuk instability dominates in the region of temperature 
T > 0.05t at small hole doping {6 < 0.2), and for negative t' > —t/i the Pomeranchuk instability dominates in all 
temperature region at the electron concentrations around the Van Hove filling. It is the leading instability at large 
negative t' < —t/3 in the region of electron density above the Van Hove filling and below some doping; for the value 
t' — —5t/12 this value of doping equals S ~ 0.2. The band splitting phase with staggered p- wave symmetry is developed 
in the region of electron concentration around half-filling, and is one of the strongest in that region. This instability 
dominates when the electron concentration n > 1.10 and t' ^ 0. The negative next-nearest-neighbors hopping 
favors this phase to be dominant instability at electron doping. The dj.2_y2--wa,Ye superconductivity dominates at low 
temperatures in certain regions of electron concentration around half-filling which depend on the value of t' ^ 0. Even 
large values of \t'\ do not destroy the dominant low-temperature behavior of dj.2_j,2-wave superconductivity at hole 
doping. It is not destroyed also at sufficiently large hole doping. However, this dominant behavior is destroyed around 
the Van Hove filling where other phases appear to dominate. At small electron doping c?-wave superconductivity 
dominates at low temperatures also, but as a result of electron-hole asymmetry of the 2D i — <' Hubbard model the 
d-wave superconductivity appears to dominate within wider region of hole doping in comparison with the electron 
doping for negative values oft'. Triplet and singlet flux phases, which have degenerate critical temperatures at t' = 0, 
occur more easily at the electron concentrations which are slightly above the Van Hove filling for t' ^ 0. For non-zero 
values of the next-nearest-neighbors hopping the critical transition temperatures of these phases are different, and 
the triplet one is higher. The triplet analog of flux phase dominates at low temperatures and t' = — 5t/12 when 
the electron concentration is slightly above the Van Hove filling. The triplet flux phase is also one of the leading 
instabilities for various values of next-nearest-neighbors hopping t' and certain region of electron concentrations. At 
t' < —t/3 and the Van Hove filling the leading instabilities are a g+ wave Pomeranchuk instability and p-wave particle- 
hole instability in triplet channel (phase of magnetic currents) at temperatures T < O.lSt, and s*-magnetic phase for 
T > 0.15i. The magnetic currents phase dominates also at very low temperatures and electron concentrations which 
are slightly smaller the Van Hove filling for t' = —5t/12, and in the temperature region T < 0.15t below the Van Hove 
density for t' = —t/3. The s*-magnetic phase is reduced strongly at low temperatures and shows a reentrant behavior 
in certain region of U/t. We have found other weaker instabilities also. Most instabilities develop at U > At, which 
are not small values. Therefore, flow equation calculations beyond second order would be desirable. Nevertheless, as 
we have found most commonly discussed types of order, and since some effects obtained in the intermediate to strong 
couplings are reproduced reasonably well by means of the flow equations, we suggest that our calculations give an 
estimate of the most important instabilities. 
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FIG. 1: Temperature phase diagram of the 2D t — t' Hubbard model for n = 1, t' — 0. Chemical potential fi — 0. SC stands 
for superconductivity, FP for flux phase, BS for band splitting, PI for Pomeranchuk instability, AF for antiferromagnetism. 




FIG. 2: Temperature phase diagram of the 2D t — t' Hubbard model for n — 0.94, t' — 0. Chemical potential varies between 
fi/t = —(0.097 ~ 0.146). Notations are the same as in Fig.0 
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FIG. 3: Temperature phase diagram of the 2D t ~ t' Hubbard model for t' — —t/3, and n — 0.68 (the Van Hove fiUing). 
Chemical potential varies between ^/t = —(1.317 ~ 1.339). TFP stands for triplet flux phase, M for magnetic particle-hole 
instability in triplet channel, other notations are the same as in Fig. Q 




FIG. 4: Temperature phase diagram of the 2D t — t' Hubbard model for t' — —5t/12, and n — 0.90. Chemical potential varies 
between /x/t = —(1.109 -i- 0.985). Notations are the same as in Fig. |21 
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FIG. 5: Temperature phase diagram of the 2D t — t' Hubbard model for t' — — 5t/12, and n = 0.55 (the Van Hove fiUing). 
Chemical potential varies between fi/t = —(1.666 ~ 1.632). Notations are the same as in Fig.O 
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FIG. 6: Temperature phase diagram of the 2D t — t' Hubbard model for t' — — 5t/12, and n = 0.70 (above the Van Hove 
filling). Chemical potential varies between ^/t = —(1.493 -t- 1.380). Notations are the same as in Fig. |3] 




FIG. 7: Temperature phase diagram of the 2D t — t' Hubbard model for t' — —t/6, and n = 1.15. Chemical potential varies 
between /x/t = —(0.001 0.080). Notations are the same as in Fig. 
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FIG. 8: Temperature phase diagram of the 2D t ~ t' Hubbard model for t' — — 5t/12, and n — 0.60 (slightly above the Van 
Hove filling). Chemical potential varies between /i/t = —(1.621 1.550). Notations are the same as in Fig. 13 
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FIG. 9: Temperature phase diagram of the 2D t ~ t' Hubbard model for t' — — 5t/12, and n — 0.50 (slightly below the Van 
Hove filling). Chemical potential varies between ^i/t = —(1.709 1.713). Notations are the same as in Fig.|3] 
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FIG. 10: Temperature phase diagram of the 2D t — t' Hubbard model for t' = —5t/12, and n = 0.35 (below the Van Hove 
filling). Chemical potential varies between ^/t = —(1.885 -^ 1.963). Notations are the same as in Fig.^l 



